<html>
<head><meta charset="utf-8"><title>Mixing sync and async · wg-async-foundations · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/index.html">wg-async-foundations</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html">Mixing sync and async</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="245506021"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/245506021" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#245506021">(Jul 09 2021 at 21:42)</a>:</h4>
<p>I'm realizing that I don't think the sketch of APIs I've painted really covers mixing sync and async very well. I imagine scopes may help here though, even though it doesn't eliminate one of the key problems. In other words, today, when you wind up with an async-sync-async sandwich, you have two problems:</p>
<ol>
<li>You have to go and insert a <code>spawn_blocking</code> call <em>before</em> the sync code, which may not be in your control (iow, typically you hit this problem you are adding the 2nd async call, but to fix it, you have to modify the 1st one).</li>
<li>Sometimes you can't insert <code>spawn_blocking</code>, because the sync code may need references or other things, and <code>spawn_blocking</code> requires <code>'static</code>.</li>
</ol>



<a name="245506043"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/245506043" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#245506043">(Jul 09 2021 at 21:42)</a>:</h4>
<p>Scoped APIs help with the 2nd one</p>



<a name="245506050"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/245506050" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#245506050">(Jul 09 2021 at 21:42)</a>:</h4>
<p>But not the 1st</p>



<a name="245506104"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/245506104" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#245506104">(Jul 09 2021 at 21:43)</a>:</h4>
<p>Tokio's block-in-place style API helps with the 1st, but comes with various gotchas (<span class="user-mention" data-user-id="243965">@Sean McArthur</span> put some energy into thinking about this...)</p>



<a name="245506141"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/245506141" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#245506141">(Jul 09 2021 at 21:43)</a>:</h4>
<p>I'm not sure we will find a fix for that?</p>



<a name="245506187"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/245506187" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#245506187">(Jul 09 2021 at 21:43)</a>:</h4>
<p>Apart from the crazy stack editing tricks that will probably not work :)</p>



<a name="245984283"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/245984283" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Carl Lerche <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#245984283">(Jul 14 2021 at 16:30)</a>:</h4>
<p>I'm not following point 1</p>



<a name="245984617"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/245984617" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#245984617">(Jul 14 2021 at 16:32)</a>:</h4>
<p>Me neither. It seems like we can have a <code>spawn_blocking</code> call which returns an <code>Async</code>, which makes sure the scope around it waits for it to complete. Since the blocking method runs to completion anyway, that isn't an unreasonable thing to do?</p>



<a name="245984718"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/245984718" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#245984718">(Jul 14 2021 at 16:33)</a>:</h4>
<p>I would rather forget about <code>block_in_place</code>, since it purely works with a multithreaded work-stealing runtime and not other async environments, like the usually more common N tasks on a single thread environment</p>



<a name="246249614"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246249614" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246249614">(Jul 16 2021 at 16:55)</a>:</h4>
<p>Let me make it concrete</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">fn_a</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="n">fn_b</span><span class="p">();</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>

<span class="k">fn</span> <span class="nf">fn_b</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="n">fn_c</span><span class="p">().</span><span class="k">await</span><span class="p">;</span><span class="w"> </span><span class="c1">// error</span>
<span class="p">}</span><span class="w"></span>

<span class="k">async</span><span class="w"> </span><span class="k">fn</span> <span class="nf">fn_c</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<p>If you have this setup, how can you resolve <code>error</code>? In particular, how can you resolve <code>error</code> if <code>fn_a</code> cannot be modified (e.g., it's in a library you are using). The usual solutions don't work:</p>
<ul>
<li>Make <code>fn_b</code> async -- that requires modifying <code>fn_a</code> to await</li>
<li>Make <code>fn_a</code> use <code>spawn_blocking</code>, and then make <code>fn_b</code> use <code>tokio::block_on</code> or something like that --- again requires modifying <code>fn_a</code></li>
</ul>
<p>As far as I know, you're just kind of stuck.</p>



<a name="246251275"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246251275" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Ibraheem Ahmed <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246251275">(Jul 16 2021 at 17:08)</a>:</h4>
<p>Well if the library with <code>fn_a</code> wants a synchronous function to run, then you can just <code>block_on</code> inside of <code>fn_b</code></p>



<a name="246255484"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246255484" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246255484">(Jul 16 2021 at 17:40)</a>:</h4>
<p>you can't, though</p>



<a name="246255510"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246255510" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246255510">(Jul 16 2021 at 17:40)</a>:</h4>
<p>bad things happen :)</p>



<a name="246256604"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246256604" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246256604">(Jul 16 2021 at 17:49)</a>:</h4>
<p>so this is about breaking API changes if you are the owner of <code>fn_b</code>?</p>



<a name="246256709"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246256709" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246256709">(Jul 16 2021 at 17:50)</a>:</h4>
<p>The problem definitely comes up - but I don't see how any of the of the proposals we discussed so far would influence that</p>



<a name="246256844"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246256844" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246256844">(Jul 16 2021 at 17:51)</a>:</h4>
<p>I think Niko's async-as-monomorphization-mode idea might :)</p>



<a name="246256867"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246256867" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246256867">(Jul 16 2021 at 17:51)</a>:</h4>
<p>but there are a lot of problems to resolve with that</p>



<a name="246257269"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246257269" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246257269">(Jul 16 2021 at 17:54)</a>:</h4>
<p><span class="user-mention silent" data-user-id="204219">Matthias247</span> <a href="#narrow/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async/near/246256709">said</a>:</p>
<blockquote>
<p>The problem definitely comes up - but I don't see how any of the of the proposals we discussed so far would influence that</p>
</blockquote>
<p>this is my point</p>



<a name="246257280"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246257280" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246257280">(Jul 16 2021 at 17:54)</a>:</h4>
<p>they don't :)</p>



<a name="246257298"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246257298" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246257298">(Jul 16 2021 at 17:55)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116009">nikomatsakis</span> <a href="#narrow/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async/near/245506021">said</a>:</p>
<blockquote>
<p>I'm realizing that I don't think the sketch of APIs I've painted really covers mixing sync and async very well. </p>
</blockquote>



<a name="246257313"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246257313" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246257313">(Jul 16 2021 at 17:55)</a>:</h4>
<p>I'm just pointing that out</p>



<a name="246264126"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246264126" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246264126">(Jul 16 2021 at 18:53)</a>:</h4>
<p>I think the only solutions which do that at the moment are ones which are using stackful coroutines, or ones which are "programming like that" (e.g. Java Project Loom which has lightweight threads that program like stackful coroutines but are still actually heap allocated continuations).</p>



<a name="246264470"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246264470" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246264470">(Jul 16 2021 at 18:57)</a>:</h4>
<p>I'm not sure if you want to have no indications of async at all - because it fully hides the fact whether you can or can not block inside a method. It works for Java because they schedule everything on a multithreaded work-stealing scheduler - so even if you are blocking since you don't know you are in an async context its not end of the world. With Rust where some people might prefer single-threaded schedulers that might not be the case?</p>



<a name="246269431"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246269431" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoey <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246269431">(Jul 16 2021 at 19:42)</a>:</h4>
<blockquote>
<p>It works for Java because they schedule everything on a multithreaded work-stealing scheduler</p>
</blockquote>
<p>Having lower level control is one of the features that makes me prefer Rust's futures system- but it does bring to mind whether or not "can safely block" should be represented at the type system level somehow.</p>



<a name="246334817"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246334817" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246334817">(Jul 17 2021 at 17:28)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> Comments on <a href="https://nikomatsakis.github.io/wg-async-foundations/vision/shiny_future/users_manual.html#bridging-the-sync-and-async-worlds">https://nikomatsakis.github.io/wg-async-foundations/vision/shiny_future/users_manual.html#bridging-the-sync-and-async-worlds</a> since it also belong to the topic of sync and async</p>
<div class="codehilite"><pre><span></span><code>std::async::blocking(|| ...).await;
</code></pre></div>
<blockquote>
<p>Note that blocking is an async function. Interally, it is built on the scope method spawn_blocking, which spawns out a task into an inner scope (deliv_scope_api):</p>
</blockquote>
<div class="codehilite"><pre><span></span><code>async fn blocking&lt;R&gt;(f: impl FnOnce() -&gt; R) -&gt; R {
    scope(|s| s.spawn_blocking(f).await).await
}
</code></pre></div>
<p>What would we gain from having another <code>scope</code> here?  The method should certainly be bound to a scope, but why wouldn't it be the "current scope" but a "child scope". Is it so that <code>Spawn</code> has to be ony implemented by <code>Scope</code> and not <code>Context</code>? But even in that case - <code>blocking</code> would have access to the current <code>Scope</code> via <code>Context</code> - otherwise it couldn't create a child scope.</p>



<a name="246335044"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Mixing%20sync%20and%20async/near/246335044" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Mixing.20sync.20and.20async.html#246335044">(Jul 17 2021 at 17:35)</a>:</h4>
<p>I think might fit into this section and what we should make a goal is that we are able to propagate cancellation around an arbitary amount of sync-&gt;async-&gt;etc contexts. E.g. I would like to see being able to do the following: </p>
<div class="codehilite"><pre><span></span><code>async fn happy_mix_and_match() {
    std::async::blocking(|| {
        crossbeam::scope(|s| {
            s.spawn(|_| {
                tokio::Runtime::builder().build().block_on(async {
                    std::async_thread::scope(async |s| {
                        for 0 .. 100 {
                            s.spawn(async {
                                std::task::sleep(Duration::from_years(123456789)).await;
                            });
                        }
                    });
                });
            });
        }).unwrap()
    }).await
}
</code></pre></div>
<p>Where cancelling <code>do_something</code> would get the cancellation signal up to the innermost timer, and thereby allows the outer async scope to exit even while it is waiting on a synchronous subtask.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>